You were prepared for gratitude, a commendation from the Admiral, your own department, parades in your name. You were also prepared to hear that your ‘list of helpful suggestions for ensuring supply ships survive random encounters’ was an impudent insult to the collective intellect of High Command, and receive a public execution for your trouble. What you weren’t prepared for was what happened: being allocated a modest stipend, assigned to a vessel, and told that if you’re so clever you should implement your plans personally.

You have 100gp to spend, and your options are as follows:

Once again, I’ve compiled some statistics on color trends in the spring/summer 2025 ready-to-wear fashion collections!

Background and Methodology

We just got done with “fashion month”, the flurry of activity in the fall when fashion designers release their spring/summer collections for the coming year and display them on runways in New York, London, Milan, and Paris.^[1]

Vogue Magazine generously shares many images from the collections — by my count, 13,570 photographs in all, typically each of a different outfit.^[2]

My question is a simple one: which colors are most common in the SS25^[3] collections in aggregate? And how do they compare to previous years? Are there changing trends in color popularity?

There’s an obvious and boring answer: the most popular color in clothing is always black. Followed by white. But things get more...

Last week, ARC released a paper called Towards a Law of Iterated Expectations for Heuristic Estimators, which follows up on previous work on formalizing the presumption of independence. Most of the work described here was done in 2023.

A brief table of contents for this post:

• What is a heuristic estimator? (One example and three analogies.)
• How might heuristic estimators help with understanding neural networks? (Three potential applications.)
• Formalizing the principle of unpredictable errors for heuristic estimation (the technical meat of the paper).

In "Formalizing the Presumption of Independence", we defined a heuristic estimator to be a hypothetical algorithm that estimates the values of mathematical expression based on arguments. That is, a heuristic estimator is an algorithm $G$ that takes as input

• A formally specified real-valued expression $Y$; and
• A set of formal "arguments" ${\pi }_1,\dots ,{\pi }_m$ --

-- and outputs an...

Short Summary

LLMs may be fundamentally incapable of fully general reasoning, and if so, short timelines are less plausible.

Longer summary

There is ML research suggesting that LLMs fail badly on attempts at general reasoning, such as planning problems, scheduling, and attempts to solve novel visual puzzles. This post provides a brief introduction to that research, and asks:

• Whether this limitation is illusory or actually exists.
• If it exists, whether it will be solved by scaling or is a problem fundamental to LLMs.
• If fundamental, whether it can be overcome by scaffolding & tooling.

If this is a real and fundamental limitation that can't be fully overcome by scaffolding, we should be skeptical of arguments like Leopold Aschenbrenner's (in his recent 'Situational Awareness') that we can just 'follow straight lines on graphs' and expect AGI...

As part of our Summer 2024 Program, MATS ran a series of discussion groups focused on questions and topics we believe are relevant to prioritizing research into AI safety. Each weekly session focused on one overarching question, and was accompanied by readings and suggested discussion questions. The purpose of running these discussions was to increase scholars’ knowledge about the AI safety ecosystem and models of how AI could cause a catastrophe, and hone scholars’ ability to think critically about threat models—ultimately, in service of helping scholars become excellent researchers.

The readings and questions were largely based on the curriculum from the Winter 2023-24 Program, with two changes:

• We reduced the number of weeks, since in the previous cohort scholars found it harder to devote time to discussion groups later

I interact with journalists quite a lot and I have specific preferences. Not just for articles, but for behaviour. And journalists do behave pretty strangely at times. 

This account comes from talking to journalists on ~10 occasions. Including being quoted in ~5 articles. 

Privacy

I do not trust journalists to abide by norms of privacy. If I talk to a friend and without asking, share what they said, with their name attached, I expect they'd be upset. But journalists regularly act as if their profession sets up the opposite norm - that everything is publishable, unless explicitly agreed otherwise. This is bizarre to me. It's like they have taken a public oath to be untrustworthy.

Perhaps they would argue that it’s a few bad journalists who behave like this, but how...

Janus, author of Simulators, has a blog with several posts about LLMs which aren't on LessWrong, but which have provided me lots of insight. This one, "Language models are multiverse generators," lays out an analogy (and playful metaphysical frame) in which language models are like laws driving the evolution of Everettian quantum multiverses. It also introduces the now-well-established Loom tool for exploring LLMs, which was apparently partly inspired by this analogy. I'm not sure why Janus didn't post this here; I think it deserves attention from LessWrong.

Other bangers from Janus' blog include Methods of prompt programming, HPMOR 32.5, Prophecies, and Surface Tension.

I think consequentialism describes only a subset of my wishes. For example, maximizing money is well modeled by it. But when I'm playing with something, it's mostly about the process, not the end result. Or when I want to respect the wishes of other people, I don't really know what end result I'm aiming for, but I can say what I'm willing or unwilling to do.

If I try to shoehorn everything into consequentialism, then I end up looking for "consequentialist permission" to do stuff. Like climbing a mountain: consequentialism says "I can put you on top of the mountain! Oh, that's not what you want? Then I can give you the feeling of having climbed it! You don't want that either? Then this is tricky..." This seems...